Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rollup of 9 pull requests #61758

Merged
merged 41 commits into from
Jun 12, 2019
Merged

Rollup of 9 pull requests #61758

merged 41 commits into from
Jun 12, 2019

Conversation

Centril
Copy link
Contributor

@Centril Centril commented Jun 12, 2019

Successful merges:

Failed merges:

r? @ghost

m-ou-se and others added 30 commits June 1, 2019 09:14
This should be used when trying to get at subsets of a larger span,
especially when the larger span is not available in the code attempting
to work with those subsets (especially common in the fmt_macros crate).

This is usually a good replacement for (BytePos, BytePos) and (usize,
usize) tuples.

This commit also removes from_inner_byte_pos, since it took usize
arguments, which is error prone.
We mistakenly pass cxxflags from the configuration to LLVM build as
CMAKE_C_FLAGS.
...and are only included in a single variant.
This commit adds a script which we'll execute on Azure Pipelines which
is intended to run in the background and passively collect CPU usage
statistics for our builders. The intention here is that we can use this
information over time to diagnose issues with builders, see where we can
optimize our build, fix parallelism issues, etc. This might not end up
being too useful in the long run but it's data we've wanted to collect
for quite some time now, so here's a stab at it!

Comments about how this is intended to work can be found in the python
script used here to collect CPU usage statistics.

Closes rust-lang#48828
RalfJung and others added 11 commits June 11, 2019 19:46
also consistently mark all these is_* methods for inlining
…ddyb

Generator optimization: Overlap locals that never have storage live at the same time

The specific goal of this optimization is to optimize async fns which use `await!`. Notably, `await!` has an enclosing scope around the futures it awaits ([definition](https://github.com/rust-lang/rust/blob/08bfe16129b0621bc90184f8704523d4929695ef/src/libstd/macros.rs#L365-L381)), which we rely on to implement the optimization.

More generally, the optimization allows overlapping the storage of some locals which are never storage-live at the same time. **We care about storage-liveness when computing the layout, because knowing a field is `StorageDead` is the only way to prove it will not be accessed, either directly or through a reference.**

To determine whether we can overlap two locals in the generator layout, we look at whether they might *both* be `StorageLive` at any point in the MIR. We use the `MaybeStorageLive` dataflow analysis for this. We iterate over every location in the MIR, and build a bitset for each local of the locals it might potentially conflict with.

Next, we assign every saved local to one or more variants. The variants correspond to suspension points, and we include the set of locals live across a given suspension point in the variant. (Note that we use liveness instead of storage-liveness here; this ensures that the local has actually been initialized in each variant it has been included in. If the local is not live across a suspension point, then it doesn't need to be included in that variant.). It's important to note that the variants are a "view" into our layout.

For the layout computation, we use a simplified approach.

1. Start with the set of locals assigned to only one variant. The rest are disqualified.
2. For each pair of locals which may conflict *and are not assigned to the same variant*, we pick one local to disqualify from overlapping.

Disqualified locals go into a non-overlapping "prefix" at the beginning of our layout. This means they always have space reserved for them. All the locals that are allowed to overlap in each variant are then laid out after this prefix, in the "overlap zone".

So, if A and B were disqualified, and X, Y, and Z were all eligible for overlap, our generator might look something like this:

You can think of a generator as an enum, where some fields are shared between variants. e.g.

```rust
enum Generator {
  Unresumed,
  Poisoned,
  Returned,
  Suspend0(A, B, X),
  Suspend1(B),
  Suspend2(A, Y, Z),
}
```

where every mention of `A` and `B` refer to the same field, which does not move when changing variants. Note that `A` and `B` would automatically be sent to the prefix in this example. Assuming that `X` is never `StorageLive` at the same time as either `Y` or `Z`, it would be allowed to overlap with them.

Note that if two locals (`Y` and `Z` in this case) are assigned to the same variant in our generator, their memory would never overlap in the layout. Thus they can both be eligible for the overlapping section, even if they are storage-live at the same time.

---

Depends on:
- [x] rust-lang#59897 Multi-variant layouts for generators
- [x] rust-lang#60840 Preserve local scopes in generator MIR
- [x] rust-lang#61373 Emit StorageDead along unwind paths for generators

Before merging:

- [x] ~Wrap the types of all generator fields in `MaybeUninitialized` in layout::ty::field~ (opened rust-lang#60889)
- [x] Make PR description more complete (e.g. explain why storage liveness is important and why we have to check every location)
- [x] Clean up TODO
- [x] Fix the layout code to enforce that the same field never moves around in the generator
- [x] Add tests for async/await
- [x] ~Reduce # bits we store by half, since the conflict relation is symmetric~ (note: decided not to do this, for simplicity)
- [x] Store liveness information for each yield point in our `GeneratorLayout`, that way we can emit more useful debuginfo AND tell miri which fields are definitely initialized for a given variant (see discussion at rust-lang#59897 (comment))
…odrAus

Implement Clone::clone_from for Option and Result

See rust-lang#28481
…r=estebank

Use Symbol, Span in libfmt_macros

I'm not super happy with this, personally, but I think it might be a decent start -- happy to take suggestions as to how to expand this or change things further.

r? @estebank

Fixes rust-lang#60795
…=pietroalbini

ci: Collect CPU usage statistics on Azure

This commit adds a script which we'll execute on Azure Pipelines which
is intended to run in the background and passively collect CPU usage
statistics for our builders. The intention here is that we can use this
information over time to diagnose issues with builders, see where we can
optimize our build, fix parallelism issues, etc. This might not end up
being too useful in the long run but it's data we've wanted to collect
for quite some time now, so here's a stab at it!

Comments about how this is intended to work can be found in the python
script used here to collect CPU usage statistics.

Closes rust-lang#48828
…n_rustc, r=oli-obk,Centril

use pattern matching for slices destructuring

refs rust-lang#61542

Use slices pattern where it seems to make sense .
implement nth_back for Range(Inclusive)

This is part of  rust-lang#54054.
is_fp and is_floating_point do the same thing, remove the former

also consistently mark all these `is_*` methods for inlining
Pass cflags rather than cxxflags to LLVM as CMAKE_C_FLAGS

We mistakenly pass cxxflags from the configuration to LLVM build as
CMAKE_C_FLAGS.
@Centril
Copy link
Contributor Author

Centril commented Jun 12, 2019

@bors r+ p=9 rollup=never

@bors
Copy link
Contributor

bors commented Jun 12, 2019

📌 Commit 4748b5b has been approved by Centril

@bors bors added the S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. label Jun 12, 2019
@bors
Copy link
Contributor

bors commented Jun 12, 2019

⌛ Testing commit 4748b5b with merge c4797fa...

bors added a commit that referenced this pull request Jun 12, 2019
Rollup of 9 pull requests

Successful merges:

 - #60187 (Generator optimization: Overlap locals that never have storage live at the same time)
 - #61348 (Implement Clone::clone_from for Option and Result)
 - #61568 (Use Symbol, Span in libfmt_macros)
 - #61632 (ci: Collect CPU usage statistics on Azure)
 - #61654 (use pattern matching for slices destructuring)
 - #61671 (implement nth_back for Range(Inclusive))
 - #61688 (is_fp and is_floating_point do the same thing, remove the former)
 - #61705 (Pass cflags rather than cxxflags to LLVM as CMAKE_C_FLAGS)
 - #61734 (Migrate rust-by-example to MdBook2)

Failed merges:

r? @ghost
@tesuji
Copy link
Contributor

tesuji commented Jun 12, 2019

#60187 has been merged (conflict-prone might happen).

@bors
Copy link
Contributor

bors commented Jun 12, 2019

☀️ Test successful - checks-travis, status-appveyor
Approved by: Centril
Pushing c4797fa to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Jun 12, 2019
@bors bors merged commit 4748b5b into rust-lang:master Jun 12, 2019
@rust-highfive
Copy link
Collaborator

📣 Toolstate changed by #61758!

Tested on commit c4797fa.
Direct link to PR: #61758

💔 rls on linux: test-pass → test-fail (cc @Xanewok, @rust-lang/infra).

rust-highfive added a commit to rust-lang-nursery/rust-toolstate that referenced this pull request Jun 12, 2019
Tested on commit rust-lang/rust@c4797fa.
Direct link to PR: <rust-lang/rust#61758>

💔 rls on linux: test-pass → test-fail (cc @Xanewok, @rust-lang/infra).
@Centril Centril deleted the rollup-ew2uxng branch June 12, 2019 13:41
@Centril Centril added the rollup A PR which is a rollup label Oct 2, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
merged-by-bors This PR was explicitly merged by bors. rollup A PR which is a rollup S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
Development

Successfully merging this pull request may close these issues.